Firewall offloading

ABSTRACT

A firewall system provides two network paths for network flows: one path through a firewall on a host device and another path through an alternative hardware or software system that handles network flows that have been analyzed and allowed by the firewall. The firewall system can then transfer network flows between the two paths according to the status of each network flow.

CROSS-REFERENCE TO RELATED APPLICATIONS

This application claims priority to U.S. Provisional Application No.63/177,018 filed on Apr. 20, 2021, the entire contents of which arehereby incorporated by reference.

FIELD

The present disclosure relates to network communications.

BACKGROUND

Firewalls can usefully monitor network traffic at various locationswithin an enterprise network and apply rules to network traffic in orderto improve or maintain network security. However, as firewall rules andsecurity countermeasures become increasingly complex, there remains aneed for a firewall system that offloads monitoring of network flows bya firewall to other networking hardware.

SUMMARY

A firewall system provides two network paths for network flows: one paththrough a firewall on a host device and another path through analternative hardware or software system that handles network flows thathave been analyzed and allowed by the firewall. The firewall system canthen transfer network flows between the two paths according to thestatus of each network flow.

BRIEF DESCRIPTION OF THE DRAWINGS

The foregoing and other objects, features and advantages of the devices,systems, and methods described herein will be apparent from thefollowing description of particular embodiments thereof, as illustratedin the accompanying drawings. The drawings are not necessarily to scale,emphasis instead being placed upon illustrating the principles of thedevices, systems, and methods described herein.

FIG. 1 depicts a block diagram of a threat management system.

FIG. 2 depicts a block diagram of a threat management system.

FIG. 3 shows a system for enterprise network threat detection.

FIG. 4 illustrates a threat management system.

FIG. 5 shows a firewall system.

FIG. 6 shows a method for operating a firewall system.

FIG. 7 shows an architecture for a firewall.

FIG. 8 illustrates interprocessor procedure calls stored in a sharedmemory.

FIG. 9 illustrates messaging for an interprocessor procedure call.

FIG. 10 is a flow chart of a method for an interprocessor procedurecall.

FIG. 11 illustrates messaging for batched procedure calls.

DETAILED DESCRIPTION

Embodiments will now be described with reference to the accompanyingfigures. The foregoing may, however, be embodied in many different formsand should not be construed as limited to the illustrated embodimentsset forth herein.

All documents mentioned herein are hereby incorporated by reference intheir entirety. References to items in the singular should be understoodto include items in the plural, and vice versa, unless explicitly statedotherwise or clear from the text. Grammatical conjunctions are intendedto express any and all disjunctive and conjunctive combinations ofconjoined clauses, sentences, words, and the like, unless otherwisestated or clear from the context. Thus, the term “or” should generallybe understood to mean “and/or” and so forth.

Recitation of ranges of values herein are not intended to be limiting,referring instead individually to any and all values falling within therange, unless otherwise indicated herein, and each separate value withinsuch a range is incorporated into the specification as if it wereindividually recited herein. The words “about,” “approximately” or thelike, when accompanying a numerical value, are to be construed asindicating a deviation as would be appreciated by one of ordinary skillin the art to operate satisfactorily for an intended purpose. Similarly,words of approximation such as “approximately” or “substantially” whenused in reference to physical characteristics, should be understood tocontemplate a range of deviations that would be appreciated by one ofordinary skill in the art to operate satisfactorily for a correspondinguse, function, purpose, or the like. Ranges of values and/or numericvalues are provided herein as examples only, and do not constitute alimitation on the scope of the described embodiments. Where ranges ofvalues are provided, they are also intended to include each value withinthe range as if set forth individually, unless expressly stated to thecontrary. The use of any and all examples, or exemplary language(“e.g.,” “such as,” or the like) provided herein, is intended merely tobetter illuminate the embodiments and does not pose a limitation on thescope of the embodiments. No language in the specification should beconstrued as indicating any unclaimed element as essential to thepractice of the embodiments.

In the following description, it is understood that terms such as“first,” “second,” “top,” “bottom,” “up,” “down,” and the like, arewords of convenience and are not to be construed as limiting terms.

FIG. 1 depicts a block diagram of a threat management system 101providing protection against a plurality of threats, such as malware,viruses, spyware, cryptoware, adware, Trojans, spam, intrusion, policyabuse, improper configuration, vulnerabilities, improper access,uncontrolled access, and more. A threat management facility 100 maycommunicate with, coordinate, and control operation of securityfunctionality at different control points, layers, and levels within thesystem 101. A number of capabilities may be provided by a threatmanagement facility 100, with an overall goal to intelligently use thebreadth and depth of information that is available about the operationand activity of compute instances and networks as well as a variety ofavailable controls. Another overall goal is to provide protection neededby an organization that is dynamic and able to adapt to changes incompute instances and new threats. In embodiments, the threat managementfacility 100 may provide protection from a variety of threats to avariety of compute instances in a variety of locations and networkconfigurations.

Just as one example, users of the threat management facility 100 maydefine and enforce policies that control access to and use of computeinstances, networks and data. Administrators may update policies such asby designating authorized users and conditions for use and access. Thethreat management facility 100 may update and enforce those policies atvarious levels of control that are available, such as by directingcompute instances to control the network traffic that is allowed totraverse firewalls and wireless access points, applications and dataavailable from servers, applications and data permitted to be accessedby endpoints, and network resources and data permitted to be run andused by endpoints. The threat management facility 100 may provide manydifferent services, and policy management may be offered as one of theservices.

Turning to a description of certain capabilities and components of thethreat management system 101, an exemplary enterprise facility 102 maybe or may include any networked computer-based infrastructure. Forexample, the enterprise facility 102 may be corporate, commercial,organizational, educational, governmental, or the like. As home networksget more complicated, and include more compute instances at home and inthe cloud, an enterprise facility 102 may also or instead include apersonal network such as a home or a group of homes. The enterprisefacility's 102 computer network may be distributed amongst a pluralityof physical premises such as buildings on a campus, and located in oneor in a plurality of geographical locations. The configuration of theenterprise facility as shown is merely exemplary, and it will beunderstood that there may be any number of compute instances, less ormore of each type of compute instances, and other types of computeinstances. As shown, the exemplary enterprise facility includes afirewall 10, a wireless access point 11, an endpoint 12, a server 14, amobile device 16, an appliance or TOT device 18, a cloud computinginstance 19, and a server 20. Again, the compute instances 10-20depicted are exemplary, and there may be any number or types of computeinstances 10-20 in a given enterprise facility. For example, in additionto the elements depicted in the enterprise facility 102, there may beone or more gateways, bridges, wired networks, wireless networks,virtual private networks, other compute instances, and so on.

The threat management facility 100 may include certain facilities, suchas a policy management facility 112, security management facility 122,update facility 120, definitions facility 114, network access rulesfacility 124, remedial action facility 128, detection techniquesfacility 130, application protection facility 150, asset classificationfacility 160, entity model facility 162, event collection facility 164,event logging facility 166, analytics facility 168, dynamic policiesfacility 170, identity management facility 172, and marketplacemanagement facility 174, as well as other facilities. For example, theremay be a testing facility, a threat research facility, and otherfacilities. It should be understood that the threat management facility100 may be implemented in whole or in part on a number of differentcompute instances, with some parts of the threat management facility ondifferent compute instances in different locations. For example, some orall of one or more of the various facilities 100, 112-174 may beprovided as part of a security agent S that is included in softwarerunning on a compute instance 10-26 within the enterprise facility. Someor all of one or more of the facilities 100, 112-174 may be provided onthe same physical hardware or logical resource as a gateway, such as afirewall 10, or wireless access point 11. Some or all of one or more ofthe facilities may be provided on one or more cloud servers that areoperated by the enterprise or by a security service provider, such asthe cloud computing instance 109.

In embodiments, a marketplace provider 199 may make available one ormore additional facilities to the enterprise facility 102 via the threatmanagement facility 100. The marketplace provider may communicate withthe threat management facility 100 via the marketplace interfacefacility 174 to provide additional functionality or capabilities to thethreat management facility 100 and compute instances 10-26. Asnon-limiting examples, the marketplace provider 199 may be a third-partyinformation provider, such as a physical security event provider; themarketplace provider 199 may be a system provider, such as a humanresources system provider or a fraud detection system provider; themarketplace provider may be a specialized analytics provider; and so on.The marketplace provider 199, with appropriate permissions andauthorization, may receive and send events, observations, inferences,controls, convictions, policy violations, or other information to thethreat management facility. For example, the marketplace provider 199may subscribe to and receive certain events, and in response, based onthe received events and other events available to the marketplaceprovider 199, send inferences to the marketplace interface, and in turnto the analytics facility 168, which in turn may be used by the securitymanagement facility 122.

The identity provider 158 may be any remote identity management systemor the like configured to communicate with an identity managementfacility 172, e.g., to confirm identity of a user as well as provide orreceive other information about users that may be useful to protectagainst threats. In general, the identity provider may be any system orentity that creates, maintains, and manages identity information forprincipals while providing authentication services to relying partyapplications, e.g., within a federation or distributed network. Theidentity provider may, for example, offer user authentication as aservice, where other applications, such as web applications, outsourcethe user authentication step to a trusted identity provider.

In embodiments, the identity provider 158 may provide user identityinformation, such as multi-factor authentication, to a SaaS application.Centralized identity providers such as Microsoft Azure, may be used byan enterprise facility instead of maintaining separate identityinformation for each application or group of applications, and as acentralized point for integrating multifactor authentication. Inembodiments, the identity management facility 172 may communicatehygiene, or security risk information, to the identity provider 158. Theidentity management facility 172 may determine a risk score for a userbased on the events, observations, and inferences about that user andthe compute instances associated with the user. If a user is perceivedas risky, the identity management facility 172 can inform the identityprovider 158, and the identity provider 158 may take steps to addressthe potential risk, such as to confirm the identity of the user, confirmthat the user has approved the SaaS application access, remediate theuser's system, or such other steps as may be useful.

In embodiments, threat protection provided by the threat managementfacility 100 may extend beyond the network boundaries of the enterprisefacility 102 to include clients (or client facilities) such as anendpoint 22 outside the enterprise facility 102, a mobile device 26, acloud computing instance 109, or any other devices, services or the likethat use network connectivity not directly associated with or controlledby the enterprise facility 102, such as a mobile network, a public cloudnetwork, or a wireless network at a hotel or coffee shop. While threatsmay come from a variety of sources, such as from network threats,physical proximity threats, secondary location threats, the computeinstances 10-26 may be protected from threats even when a computeinstance 10-26 is not connected to the enterprise facility 102 network,such as when compute instances 22, 26 use a network that is outside ofthe enterprise facility 102 and separated from the enterprise facility102, e.g., by a gateway, a public network, and so forth.

In some implementations, compute instances 10-26 may communicate withcloud applications, such as a SaaS application 156. The SaaS application156 may be an application that is used by but not operated by theenterprise facility 102. Exemplary commercially available SaaSapplications 156 include Salesforce, Amazon Web Services (AWS)applications, Google Apps applications, Microsoft Office 365applications and so on. A given SaaS application 156 may communicatewith an identity provider 158 to verify user identity consistent withthe requirements of the enterprise facility 102. The compute instances10-26 may communicate with an unprotected server (not shown) such as aweb site or a third-party application through an internetwork 154 suchas the Internet or any other public network, private network orcombination of these.

In embodiments, aspects of the threat management facility 100 may beprovided as a stand-alone solution. In other embodiments, aspects of thethreat management facility 100 may be integrated into a third-partyproduct. An application programming interface (e.g., a source codeinterface) may be provided such that aspects of the threat managementfacility 100 may be integrated into or used by or with otherapplications. For instance, the threat management facility 100 may bestand-alone in that it provides direct threat protection to anenterprise or computer resource, where protection is subscribed todirectly 100. Alternatively, the threat management facility may offerprotection indirectly, through a third-party product, where anenterprise may subscribe to services through the third-party product,and threat protection to the enterprise may be provided by the threatmanagement facility 100 through the third-party product.

The security management facility 122 may provide protection from avariety of threats by providing, as non-limiting examples, endpointsecurity and control, email security and control, web security andcontrol, reputation-based filtering, machine learning classification,control of unauthorized users, control of guest and non-compliantcomputers, and more.

The security management facility 122 may provide malicious codeprotection to a compute instance. The security management facility 122may include functionality to scan applications, files, and data formalicious code, remove or quarantine applications and files, preventcertain actions, perform remedial actions, as well as other securitymeasures. Scanning may use any of a variety of techniques, includingwithout limitation signatures, identities, classifiers, and othersuitable scanning techniques. In embodiments, the scanning may includescanning some or all files on a periodic basis, scanning an applicationwhen the application is executed, scanning data transmitted to or from adevice, scanning in response to predetermined actions or combinations ofactions, and so forth. The scanning of applications, files, and data maybe performed to detect known or unknown malicious code or unwantedapplications. Aspects of the malicious code protection may be provided,for example, in the security agent of an endpoint 12, in a wirelessaccess point 11 or firewall 10, as part of application protection 150provided by the cloud, and so on.

In an embodiment, the security management facility 122 may provide foremail security and control, for example to target spam, viruses, spywareand phishing, to control email content, and the like. Email security andcontrol may protect against inbound and outbound threats, protect emailinfrastructure, prevent data leakage, provide spam filtering, and more.Aspects of the email security and control may be provided, for example,in the security agent of an endpoint 12, in a wireless access point 11or firewall 10, as part of application protection 150 provided by thecloud, and so on.

In an embodiment, security management facility 122 may provide for websecurity and control, for example, to detect or block viruses, spyware,malware, unwanted applications, help control web browsing, and the like,which may provide comprehensive web access control enabling safe,productive web browsing. Web security and control may provide Internetuse policies, reporting on suspect compute instances, security andcontent filtering, active monitoring of network traffic, URI filtering,and the like. Aspects of the web security and control may be provided,for example, in the security agent of an endpoint 12, in a wirelessaccess point 11 or firewall 10, as part of application protection 150provided by the cloud, and so on.

In an embodiment, the security management facility 122 may provide fornetwork access control, which generally controls access to and use ofnetwork connections. Network control may stop unauthorized, guest, ornon-compliant systems from accessing networks, and may control networktraffic that is not otherwise controlled at the client level. Inaddition, network access control may control access to virtual privatenetworks (VPN), where VPNs may, for example, include communicationsnetworks tunneled through other networks and establishing logicalconnections acting as virtual networks. In embodiments, a VPN may betreated in the same manner as a physical network. Aspects of networkaccess control may be provided, for example, in the security agent of anendpoint 12, in a wireless access point 11 or firewall 10, as part ofapplication protection 150 provided by the cloud, e.g., from the threatmanagement facility 100 or other network resource(s).

In an embodiment, the security management facility 122 may provide forhost intrusion prevention through behavioral monitoring and/or runtimemonitoring, which may guard against unknown threats by analyzingapplication behavior before or as an application runs. This may includemonitoring code behavior, application programming interface calls madeto libraries or to the operating system, or otherwise monitoringapplication activities. Monitored activities may include, for example,reading and writing to memory, reading and writing to disk, networkcommunication, process interaction, and so on. Behavior and runtimemonitoring may intervene if code is deemed to be acting in a manner thatis suspicious or malicious. Aspects of behavior and runtime monitoringmay be provided, for example, in the security agent of an endpoint 12,in a wireless access point 11 or firewall 10, as part of applicationprotection 150 provided by the cloud, and so on.

In an embodiment, the security management facility 122 may provide forreputation filtering, which may target or identify sources of knownmalware. For instance, reputation filtering may include lists of URIs ofknown sources of malware or known suspicious IP addresses, code authors,code signers, or domains, that when detected may invoke an action by thethreat management facility 100. Based on reputation, potential threatsources may be blocked, quarantined, restricted, monitored, or somecombination of these, before an exchange of data can be made. Aspects ofreputation filtering may be provided, for example, in the security agentof an endpoint 12, in a wireless access point 11 or firewall 10, as partof application protection 150 provided by the cloud, and so on. Inembodiments, some reputation information may be stored on a computeinstance 10-26, and other reputation data available through cloudlookups to an application protection lookup database, such as may beprovided by application protection 150.

In embodiments, information may be sent from the enterprise facility 102to a third party, such as a security vendor, or the like, which may leadto improved performance of the threat management facility 100. Ingeneral, feedback may be useful for any aspect of threat detection. Forexample, the types, times, and number of virus interactions that anenterprise facility 102 experiences may provide useful information forthe preventions of future virus threats. Feedback may also be associatedwith behaviors of individuals within the enterprise, such as beingassociated with most common violations of policy, network access,unauthorized application loading, unauthorized external device use, andthe like. In embodiments, feedback may enable the evaluation orprofiling of client actions that are violations of policy that mayprovide a predictive model for the improvement of enterprise policies.

An update management facility 120 may provide control over when updatesare performed. The updates may be automatically transmitted, manuallytransmitted, or some combination of these. Updates may include software,definitions, reputations or other code or data that may be useful to thevarious facilities. For example, the update facility 120 may managereceiving updates from a provider, distribution of updates to enterprisefacility 102 networks and compute instances, or the like. Inembodiments, updates may be provided to the enterprise facility's 102network, where one or more compute instances on the enterprisefacility's 102 network may distribute updates to other computeinstances.

The threat management facility 100 may include a policy managementfacility 112 that manages rules or policies for the enterprise facility102. Exemplary rules include access permissions associated withnetworks, applications, compute instances, users, content, data, and thelike. The policy management facility 112 may use a database, a textfile, other data store, or a combination to store policies. In anembodiment, a policy database may include a block list, a black list, anallowed list, a white list, and more. As a few non-limiting examples,policies may include a list of enterprise facility 102 external networklocations/applications that may or may not be accessed by computeinstances, a list of types/classifications of network locations orapplications that may or may not be accessed by compute instances, andcontextual rules to evaluate whether the lists apply. For example, theremay be a rule that does not permit access to sporting websites. When awebsite is requested by the client facility, a security managementfacility 122 may access the rules within a policy facility to determineif the requested access is related to a sporting website.

The policy management facility 112 may include access rules and policiesthat are distributed to maintain control of access by the computeinstances 10-26 to network resources. Exemplary policies may be definedfor an enterprise facility, application type, subset of applicationcapabilities, organization hierarchy, compute instance type, user type,network location, time of day, connection type, or any other suitabledefinition. Policies may be maintained through the threat managementfacility 100, in association with a third party, or the like. Forexample, a policy may restrict instant messaging (IM) activity bylimiting such activity to support personnel when communicating withcustomers. More generally, this may allow communication for departmentsas necessary or helpful for department functions, but may otherwisepreserve network bandwidth for other activities by restricting the useof IM to personnel that need access for a specific purpose. In anembodiment, the policy management facility 112 may be a stand-aloneapplication, may be part of the network server facility 142, may be partof the enterprise facility 102 network, may be part of the clientfacility, or any suitable combination of these.

The policy management facility 112 may include dynamic policies that usecontextual or other information to make security decisions. As describedherein, the dynamic policies facility 170 may generate policiesdynamically based on observations and inferences made by the analyticsfacility. The dynamic policies generated by the dynamic policy facility170 may be provided by the policy management facility 112 to thesecurity management facility 122 for enforcement.

In embodiments, the threat management facility 100 may provideconfiguration management as an aspect of the policy management facility112, the security management facility 122, or some combination.Configuration management may define acceptable or requiredconfigurations for the compute instances 10-26, applications, operatingsystems, hardware, or other assets, and manage changes to theseconfigurations. Assessment of a configuration may be made againststandard configuration policies, detection of configuration changes,remediation of improper configurations, application of newconfigurations, and so on. An enterprise facility may have a set ofstandard configuration rules and policies for particular computeinstances which may represent a desired state of the compute instance.For example, on a given compute instance 12, 14, 18, a version of aclient firewall may be required to be running and installed. If therequired version is installed but in a disabled state, the policyviolation may prevent access to data or network resources. A remediationmay be to enable the firewall. In another example, a configurationpolicy may disallow the use of USB disks, and policy management 112 mayrequire a configuration that turns off USB drive access via a registrykey of a compute instance. Aspects of configuration management may beprovided, for example, in the security agent of an endpoint 12, in awireless access point 11 or firewall 10, as part of applicationprotection 150 provided by the cloud, or any combination of these.

In embodiments, the threat management facility 100 may also provide forthe isolation or removal of certain applications that are not desired ormay interfere with the operation of a compute instance 10-26 or thethreat management facility 100, even if such application is not malwareper se. The operation of such products may be considered a configurationviolation. The removal of such products may be initiated automaticallywhenever such products are detected, or access to data and networkresources may be restricted when they are installed and running. In thecase where such applications are services which are provided indirectlythrough a third-party product, the applicable application or processesmay be suspended until action is taken to remove or disable thethird-party product.

The policy management facility 112 may also require update management(e.g., as provided by the update facility 120). Update management forthe security facility 122 and policy management facility 112 may beprovided directly by the threat management facility 100, or, forexample, by a hosted system. In embodiments, the threat managementfacility 100 may also provide for patch management, where a patch may bean update to an operating system, an application, a system tool, or thelike, where one of the reasons for the patch is to reduce vulnerabilityto threats.

In embodiments, the security facility 122 and policy management facility112 may push information to the enterprise facility 102 network and/orthe compute instances 10-26, the enterprise facility 102 network and/orcompute instances 10-26 may pull information from the security facility122 and policy management facility 112, or there may be a combination ofpushing and pulling of information. For example, the enterprise facility102 network and/or compute instances 10-26 may pull update informationfrom the security facility 122 and policy management facility 112 viathe update facility 120, an update request may be based on a timeperiod, by a certain time, by a date, on demand, or the like. In anotherexample, the security facility 122 and policy management facility 112may push the information to the enterprise facility's 102 network and/orcompute instances 10-26 by providing notification that there are updatesavailable for download and/or transmitting the information. In anembodiment, the policy management facility 112 and the security facility122 may work in concert with the update management facility 120 toprovide information to the enterprise facility's 102 network and/orcompute instances 10-26. In various embodiments, policy updates,security updates and other updates may be provided by the same ordifferent modules, which may be the same or separate from a securityagent running on one of the compute instances 10-26.

As threats are identified and characterized, the definition facility 114of the threat management facility 100 may manage definitions used todetect and remediate threats. For example, identity definitions may beused for scanning files, applications, data streams, etc. for thedetermination of malicious code. Identity definitions may includeinstructions and data that can be parsed and acted upon for recognizingfeatures of known or potentially malicious code. Definitions also mayinclude, for example, code or data to be used in a classifier, such as aneural network or other classifier that may be trained using machinelearning. Updated code or data may be used by the classifier to classifythreats. In embodiments, the threat management facility 100 and thecompute instances 10-26 may be provided with new definitionsperiodically to include most recent threats. Updating of definitions maybe managed by the update facility 120, and may be performed upon requestfrom one of the compute instances 10-26, upon a push, or somecombination. Updates may be performed upon a time period, on demand froma device 10-26, upon determination of an important new definition or anumber of definitions, and so on.

A threat research facility (not shown) may provide a continuouslyongoing effort to maintain the threat protection capabilities of thethreat management facility 100 in light of continuous generation of newor evolved forms of malware. Threat research may be provided byresearchers and analysts working on known threats, in the form ofpolicies, definitions, remedial actions, and so on.

The security management facility 122 may scan an outgoing file andverify that the outgoing file is permitted to be transmitted accordingto policies. By checking outgoing files, the security managementfacility 122 may be able discover threats that were not detected on oneof the compute instances 10-26, or policy violation, such transmittal ofinformation that should not be communicated unencrypted.

The threat management facility 100 may control access to the enterprisefacility 102 networks. A network access facility 124 may restrict accessto certain applications, networks, files, printers, servers, databases,and so on. In addition, the network access facility 124 may restrictuser access under certain conditions, such as the user's location, usagehistory, need to know, job position, connection type, time of day,method of authentication, client-system configuration, or the like.Network access policies may be provided by the policy managementfacility 112, and may be developed by the enterprise facility 102, orpre-packaged by a supplier. Network access facility 124 may determine ifa given compute instance 10-22 should be granted access to a requestednetwork location, e.g., inside or outside of the enterprise facility102. Network access facility 124 may determine if a compute instance 22,26 such as a device outside the enterprise facility 102 may access theenterprise facility 102. For example, in some cases, the policies mayrequire that when certain policy violations are detected, certainnetwork access is denied. The network access facility 124 maycommunicate remedial actions that are necessary or helpful to bring adevice back into compliance with policy as described below with respectto the remedial action facility 128. Aspects of the network accessfacility 124 may be provided, for example, in the security agent of theendpoint 12, in a wireless access point 11, in a firewall 10, as part ofapplication protection 150 provided by the cloud, and so on.

In an embodiment, the network access facility 124 may have access topolicies that include one or more of a block list, a black list, anallowed list, a white list, an unacceptable network site database, anacceptable network site database, a network site reputation database, orthe like of network access locations that may or may not be accessed bythe client facility. Additionally, the network access facility 124 mayuse rule evaluation to parse network access requests and apply policies.The network access rule facility 124 may have a generic set of policiesfor all compute instances, such as denying access to certain types ofwebsites, controlling instant messenger accesses, or the like. Ruleevaluation may include regular expression rule evaluation, or other ruleevaluation method(s) for interpreting the network access request andcomparing the interpretation to established rules for network access.Classifiers may be used, such as neural network classifiers or otherclassifiers that may be trained by machine learning.

The threat management facility 100 may include an asset classificationfacility 160. The asset classification facility will discover the assetspresent in the enterprise facility 102. A compute instance such as anyof the compute instances 10-26 described herein may be characterized asa stack of assets. The one level asset is an item of physical hardware.The compute instance may be, or may be implemented on physical hardware,and may have or may not have a hypervisor, or may be an asset managed bya hypervisor. The compute instance may have an operating system (e.g.,Windows, MacOS, Linux, Android, iOS). The compute instance may have oneor more layers of containers. The compute instance may have one or moreapplications, which may be native applications, e.g., for a physicalasset or virtual machine, or running in containers within a computingenvironment on a physical asset or virtual machine, and thoseapplications may link libraries or other code or the like, e.g., for auser interface, cryptography, communications, device drivers,mathematical or analytical functions and so forth. The stack may alsointeract with data. The stack may also or instead interact with users,and so users may be considered assets.

The threat management facility may include entity models 162. The entitymodels may be used, for example, to determine the events that aregenerated by assets. For example, some operating systems may provideuseful information for detecting or identifying events. For examples,operating systems may provide process and usage information thataccessed through an API. As another example, it may be possible toinstrument certain containers to monitor the activity of applicationsrunning on them. As another example, entity models for users may defineroles, groups, permitted activities and other attributes.

The event collection facility 164 may be used to collect events from anyof a wide variety of sensors that may provide relevant events from anasset, such as sensors on any of the compute instances 10-26, theapplication protection facility 150, a cloud computing instance 109 andso on. The events that may be collected may be determined by the entitymodels. There may be a variety of events collected. Events may include,for example, events generated by the enterprise facility 102 or thecompute instances 10-26, such as by monitoring streaming data through agateway such as firewall 10 and wireless access point 11, monitoringactivity of compute instances, monitoring stored files/data on thecompute instances 10-26 such as desktop computers, laptop computers,other mobile computing devices, and cloud computing instances 19, 109.Events may range in granularity. An exemplary event may be communicationof a specific packet over the network. Another exemplary event may beidentification of an application that is communicating over a network.

The event logging facility 166 may be used to store events collected bythe event collection facility 164. The event logging facility 166 maystore collected events so that they can be accessed and analyzed by theanalytics facility 168. Some events may be collected locally, and someevents may be communicated to an event store in a central location orcloud facility. Events may be logged in any suitable format.

Events collected by the event logging facility 166 may be used by theanalytics facility 168 to make inferences and observations about theevents. These observations and inferences may be used as part ofpolicies enforced by the security management facility Observations orinferences about events may also be logged by the event logging facility166.

When a threat or other policy violation is detected by the securitymanagement facility 122, the remedial action facility 128 may be used toremediate the threat. Remedial action may take a variety of forms,non-limiting examples including collecting additional data about thethreat, terminating or modifying an ongoing process or interaction,sending a warning to a user or administrator, downloading a data filewith commands, definitions, instructions, or the like to remediate thethreat, requesting additional information from the requesting device,such as the application that initiated the activity of interest,executing a program or application to remediate against a threat orviolation, increasing telemetry or recording interactions for subsequentevaluation, (continuing to) block requests to a particular networklocation or locations, scanning a requesting application or device,quarantine of a requesting application or the device, isolation of therequesting application or the device, deployment of a sandbox, blockingaccess to resources, e.g., a USB port, or other remedial actions. Moregenerally, the remedial action facility 122 may take any steps or deployany measures suitable for addressing a detection of a threat, potentialthreat, policy violation or other event, code or activity that mightcompromise security of a computing instance 10-26 or the enterprisefacility 102.

FIG. 2 depicts a block diagram of a threat management system 201 such asany of the threat management systems described herein, and including acloud enterprise facility 280. The cloud enterprise facility 280 mayinclude servers 284, 286, and a firewall 282. The servers 284, 286 onthe cloud enterprise facility 280 may run one or more enterpriseapplications and make them available to the enterprise facilities 102compute instances 10-26. It should be understood that there may be anynumber of servers 284, 286 and firewalls 282, as well as other computeinstances in a given cloud enterprise facility 280. It also should beunderstood that a given enterprise facility may use both SaaSapplications 156 and cloud enterprise facilities 280, or, for example, aSaaS application 156 may be deployed on a cloud enterprise facility 280.As such, the configurations in FIG. 1 and FIG. 2 are shown by way ofexamples and not exclusive alternatives.

FIG. 3 shows a system 300 for enterprise network threat detection. Thesystem 300 may use any of the various tools and techniques for threatmanagement contemplated herein. In the system, a number of endpointssuch as the endpoint 302 may log events in a data recorder 304. A localagent on the endpoint 302 such as the security agent 306 may filter thisdata and feeds a filtered data stream to a threat management facility308 such as a central threat management facility or any of the otherthreat management facilities described herein. The threat managementfacility 308 can locally or globally tune filtering by local agentsbased on the current data stream, and can query local event datarecorders for additional information where necessary or helpful inthreat detection or forensic analysis. The threat management facility308 may also or instead store and deploys a number of security toolssuch as a web-based user interface that is supported by machine learningmodels to aid in the identification and assessment of potential threatsby a human user. This may, for example, include machine learninganalysis of new code samples, models to provide human-readable contextfor evaluating potential threats, and any of the other tools ortechniques described herein. More generally, the threat managementfacility 308 may provide any of a variety of threat management tools 316to aid in the detection, evaluation, and remediation of threats orpotential threats.

The threat management facility 308 may perform a range of threatmanagement functions such as any of those described herein. The threatmanagement facility 308 may generally include an application programminginterface 310 to third party services 320, a user interface 312 foraccess to threat management and network administration functions, and anumber of threat detection tools 314.

In general, the application programming interface 310 may supportprogrammatic connections with third party services 320. The applicationprogramming interface 310 may, for example, connect to Active Directoryor other customer information about files, data storage, identities anduser profiles, roles, access privileges and so forth. More generally theapplication programming interface 310 may provide a programmaticinterface for customer or other third party context, information,administration and security tools, and so forth. The applicationprogramming interface 310 may also or instead provide a programmaticinterface for hosted applications, identity provider integration toolsor services, and so forth.

The user interface 312 may include a website or other graphicalinterface or the like, and may generally provide an interface for userinteraction with the threat management facility 308, e.g., for threatdetection, network administration, audit, configuration and so forth.This user interface 312 may generally facilitate human curation ofintermediate threats as contemplated herein, e.g., by presentingintermediate threats along with other supplemental information, andproviding controls for user to dispose of such intermediate threats asdesired, e.g., by permitting execution or access, by denying executionor access, or by engaging in remedial measures such as sandboxing,quarantining, vaccinating, and so forth.

The threat detection tools 314 may be any of the threat detection tools,algorithms, techniques or the like described herein, or any other toolsor the like useful for detecting threats or potential threats within anenterprise network. This may, for example, include signature basedtools, behavioral tools, machine learning models, and so forth. Ingeneral, the threat detection tools 314 may use event data provided byendpoints within the enterprise network, as well as any other availablecontext such as network activity, heartbeats, and so forth to detectmalicious software or potentially unsafe conditions for a network orendpoints connected to the network. In one aspect, the threat detectiontools 314 may usefully integrate event data from a number of endpoints(including, e.g., network components such as gateways, routers andfirewalls) for improved threat detection in the context of complex ordistributed threats. The threat detection tools 314 may also or insteadinclude tools for reporting to a separate modeling and analysis platform318, e.g., to support further investigation of security issues, creationor refinement of threat detection models or algorithms, review andanalysis of security breaches and so forth.

The threat management tools 316 may generally be used to manage orremediate threats to the enterprise network that have been identifiedwith the threat detection tools 314 or otherwise. Threat managementtools 316 may, for example, include tools for sandboxing, quarantining,removing, or otherwise remediating or managing malicious code ormalicious activity, e.g., using any of the techniques described herein.

The endpoint 302 may be any of the endpoints or other compute instancesor the like described herein. This may, for example, include end-usercomputing devices, mobile devices, firewalls, gateways, servers, routersand any other computing devices or instances that might connect to anenterprise network. As described above, the endpoint 302 may generallyinclude a security agent 306 that locally supports threat management onthe endpoint 302, such as by monitoring for malicious activity, managingsecurity components on the endpoint 302, maintaining policy compliance,and communicating with the threat management facility 308 to supportintegrated security protection as contemplated herein. The securityagent 306 may, for example, coordinate instrumentation of the endpoint302 to detect various event types involving various computing objects onthe endpoint 302, and supervise logging of events in a data recorder304. The security agent 306 may also or instead scan computing objectssuch as electronic communications or files, monitor behavior ofcomputing objects such as executables, and so forth. The security agent306 may, for example, apply signature-based or behavioral threatdetection techniques, machine learning models (e.g., models developed bythe modeling and analysis platform), or any other tools or the likesuitable for detecting malware or potential malware on the endpoint 302.

The data recorder 304 may log events occurring on or related to theendpoint. This may, for example, include events associated withcomputing objects on the endpoint 302 such as file manipulations,software installations, and so forth. This may also or instead includeactivities directed from the endpoint 302, such as requests for contentfrom Uniform Resource Locators or other network activity involvingremote resources. The data recorder 304 may record data at any frequencyand any level of granularity consistent with proper operation of theendpoint 302 in an intended or desired manner.

The endpoint 302 may include a filter 322 to manage a flow ofinformation from the data recorder 304 to a remote resource such as thethreat detection tools 314 of the threat management facility 308. Inthis manner, a detailed log of events may be maintained locally on eachendpoint, while network resources can be conserved for reporting of afiltered event stream that contains information believed to be mostrelevant to threat detection. The filter 322 may also or instead beconfigured to report causal information that causally relatescollections of events to one another. In general, the filter 322 may beconfigurable so that, for example, the threat management facility 308can increase or decrease the level of reporting based on a currentsecurity status of the endpoint, a group of endpoints, the enterprisenetwork and the like. The level of reporting may also or instead bebased on currently available network and computing resources, or anyother appropriate context.

In another aspect, the endpoint 302 may include a query interface 324 sothat remote resources such as the threat management facility 308 canquery the data recorder 304 remotely for additional information. Thismay include a request for specific events, activity for specificcomputing objects, or events over a specific time frame, or somecombination of these. Thus, for example, the threat management facility308 may request all changes to the registry of system information forthe past forty eight hours, all files opened by system processes in thepast day, all network connections or network communications within thepast hour, or any other parametrized request for activities monitored bythe data recorder 304. In another aspect, the entire data log, or theentire log over some predetermined window of time, may be request forfurther analysis at a remote resource.

It will be appreciated that communications among third party services320, a threat management facility 308, and one or more endpoints such asthe endpoint 302 may be facilitated by using consistent namingconventions across products and machines. For example, the system 300may usefully implement globally unique device identifiers, useridentifiers, application identifiers, data identifiers, Uniform ResourceLocators, network flows, and files. The system may also or instead usetuples to uniquely identify communications or network connections basedon, e.g., source and destination addresses and so forth.

According to the foregoing, a system disclosed herein includes anenterprise network, and endpoint coupled to the enterprise network, anda threat management facility coupled in a communicating relationshipwith the endpoint and a plurality of other endpoints through theenterprise network. The endpoint may have a data recorder that stores anevent stream of event data for computing objects, a filter for creatinga filtered event stream with a subset of event data from the eventstream, and a query interface for receiving queries to the data recorderfrom a remote resource, the endpoint further including a local securityagent configured to detect malware on the endpoint based on event datastored by the data recorder, and further configured to communicate thefiltered event stream over the enterprise network. The threat managementfacility may be configured to receive the filtered event stream from theendpoint, detect malware on the endpoint based on the filtered eventstream, and remediate the endpoint when malware is detected, the threatmanagement facility further configured to modify security functionswithin the enterprise network based on a security state of the endpoint.

The threat management facility may be configured to adjust reporting ofevent data through the filter in response to a change in the filteredevent stream received from the endpoint. The threat management facilitymay be configured to adjust reporting of event data through the filterwhen the filtered event stream indicates a compromised security state ofthe endpoint. The threat management facility may be configured to adjustreporting of event data from one or more other endpoints in response toa change in the filtered event stream received from the endpoint. Thethreat management facility may be configured to adjust reporting ofevent data through the filter when the filtered event stream indicates acompromised security state of the endpoint. The threat managementfacility may be configured to request additional data from the datarecorder when the filtered event stream indicates a compromised securitystate of the endpoint. The threat management facility may be configuredto request additional data from the data recorder when a security agentof the endpoint reports a security compromise independently from thefiltered event stream. The threat management facility may be configuredto adjust handling of network traffic at a gateway to the enterprisenetwork in response to a predetermined change in the filtered eventstream. The threat management facility may include a machine learningmodel for identifying potentially malicious activity on the endpointbased on the filtered event stream. The threat management facility maybe configured to detect potentially malicious activity based on aplurality of filtered event streams from a plurality of endpoints. Thethreat management facility may be configured to detect malware on theendpoint based on the filtered event stream and additional context forthe endpoint.

The data recorder may record one or more events from a kernel driver.The data recorder may record at least one change to a registry of systemsettings for the endpoint. The endpoints may include a server, afirewall for the enterprise network, a gateway for the enterprisenetwork, or any combination of these. The endpoint may be coupled to theenterprise network through a virtual private network or a wirelessnetwork. The endpoint may be configured to periodically transmit asnapshot of aggregated, unfiltered data from the data recorder to thethreat management facility for remote storage. The data recorder may beconfigured to delete records in the data recorder corresponding to thesnapshot in order to free memory on the endpoint for additionalrecording.

FIG. 4 illustrates a threat management system. In general, the systemmay include an endpoint 402, a firewall 404, a server 406 and a threatmanagement facility 408 coupled to one another directly or indirectlythrough a data network 405, all as generally described above. Each ofthe entities depicted in FIG. 4 may, for example, be implemented on oneor more computing devices such as the computing device described herein.A number of systems may be distributed across these various componentsto support threat detection, such as a coloring system 410, a keymanagement system 412 and a heartbeat system 414, each of which mayinclude software components executing on any of the foregoing systemcomponents, and each of which may communicate with the threat managementfacility 408 and an endpoint threat detection agent 420 executing on theendpoint 402 to support improved threat detection and remediation.

The coloring system 410 may be used to label or color software objectsfor improved tracking and detection of potentially harmful activity. Thecoloring system 410 may, for example, label files, executables,processes, network communications, data sources and so forth with anysuitable information. A variety of techniques may be used to selectstatic and/or dynamic labels for any of these various software objects,and to manage the mechanics of applying and propagating coloringinformation as appropriate. For example, a process may inherit a colorfrom an application that launches the process. Similarly, a file mayinherit a color from a process when it is created or opened by aprocess, and/or a process may inherit a color from a file that theprocess has opened. More generally, any type of labeling, as well asrules for propagating, inheriting, changing, or otherwise manipulatingsuch labels, may be used by the coloring system 410 as contemplatedherein.

The key management system 412 may support management of keys for theendpoint 402 in order to selectively permit or prevent access to contenton the endpoint 402 on a file-specific basis, a process-specific basis,an application-specific basis, a user-specific basis, or any othersuitable basis in order to prevent data leakage, and in order to supportmore fine-grained and immediate control over access to content on theendpoint 402 when a security compromise is detected. Thus, for example,if a particular process executing on the endpoint is compromised, orpotentially compromised or otherwise under suspicion, keys to thatprocess may be revoked in order to prevent, e.g., data leakage or othermalicious activity.

The heartbeat system 414 may be used to provide periodic or aperiodicinformation from the endpoint 402 or other system components aboutsystem health, security, status, and so forth. A heartbeat may beencrypted or plaintext, or some combination of these, and may becommunicated unidirectionally (e.g., from the endpoint 408 to the threatmanagement facility 408) or bidirectionally (e.g., between the endpoint402 and the server 406, or any other pair of system components) on anyuseful schedule.

In general, these various monitoring and management systems maycooperate to provide improved threat detection and response. Forexample, the coloring system 410 may be used to evaluate when aparticular process is potentially opening inappropriate files based onan inconsistency or mismatch in colors, and a potential threat may beconfirmed based on an interrupted heartbeat from the heartbeat system414. The key management system 412 may then be deployed to revoke keysto the process so that no further files can be opened, deleted orotherwise modified. More generally, the cooperation of these systemsenables a wide variety of reactive measures that can improve detectionand remediation of potential threats to an endpoint.

FIG. 5 shows a firewall system. In general, the system 500 may include ahost device 502 having a firewall 504 executing in a kernel space and anintrusion prevention system 506 executing in a user space. A networkprocessor 508 may connect network flows 510 including network packets orthe like between a source and a destination. An offload module 512 mayexecute on the network processor, and may communicate with the firewall504 through one or more application programming interfaces (indicatedgenerally by an arrow 514) so that the firewall 504 can redirect networkflows 510 to the network processor 508 and, when appropriate, theoffload module can return network flows 510 to the firewall 504. Thehost 502 may include a memory 516 storing one or more firewall rulesused by the firewall 504 in determining a firewall action for thenetwork flows 510. The network processor 508 may include a memory 518with a lookup table storing a list of one or more network flowconnections directed through the offload module 512 by the firewall 504.In this context, it will be understood that, while the network flows 510are described as passing “through” the firewall 504 or the offloadmodule 512, it is not necessary that each physical packet of data passesthrough either of these processes or modules as depicted in FIG. 5.Rather, it should be understood more generally that either the firewall504 or the offload module 512 will be responsible for each network flowassociated with each connection passing through the network processor508, or, as described in alternative embodiments below that do not usesa network processor 508, through a virtual fast path hosted in thekernel space of the host device 502.

In general, the offload module 512 may be configured as described hereinto receive firewall rules from the firewall 504, and to deploy thesefirewall rules on the network flow 510 in a manner that cooperates withor bypasses the firewall 504. In one aspect, the offload module 512 maybe deployed on separate hardware as described above to reduce processingburdens on the host device 502. In another aspect, the offload module512 may include a kernel space process or other process on the firewall504. The offload module 512 may also or instead be distributed betweenthe host device 502 and external hardware (such as the networkprocessor), and/or some other computing devices suitable for supportinghandling of the network flow 510 according to firewall rules asdescribed herein.

The host device 502 may also include an intrusion prevention system 506executing in the user space and generally configured to detect potentialthreats in the network flow 510, e.g., using any of the malware orintrusion prevention techniques described herein, or otherwise known inthe art. For example, the intrusion prevention system 506 may performantivirus scanning, web traffic filtering, secure socket layerinspection, or any other security scans or analysis applicable tonetwork traffic, including techniques using packet coloring as describedherein.

In general, the firewall 504 may apply firewall rules, e.g., accordingto a security policy for an enterprise network, to connections withinthe network flow 510, such as each new incoming connection or otherconnection created or existing within the network flows. For example,the firewall 504 may receive firewall rules or related securityinformation from a threat management facility such as any of the threatmanagement facilities described herein, which may manage the firewallrules to add, remove, or modify firewall rules according to a current oranticipated threat environment. A firewall rule may be associated withone or more firewall actions. Once a particular firewall action has beenidentified (e.g., based on a threat environment, network connection,source and destination, and the like) for a connection, the firewallaction may be communicated to the offload module 512 for use in managingone or more of the network flows 510 independently from the firewall504. When a network flow 510 appears invalid (e.g., based on theinformation communicated to the offload module 512) or is absent fromthe lookup table, or if the network flow 510 is identified aspotentially malicious by the intrusion prevention system 506, theoffload module 512 may redirect the corresponding connection back to thefirewall 504 for further processing.

The connection lookup table may identify each of one or more connectionswithin the network flow(s) 510 using any suitable connection identifierssuch as one or more of an Internet Protocol source and destinationaddress, a layer 4 source and destination address, a Medium AccessController source and destination address, and a protocol identifier.Once the firewall 504 has analyzed a network flow 510 for compliancewith firewall rules, the firewall 504 may identify a correspondingconnection (or connections) using these identifiers, which may betransmitted to the offload module 512 for storage in the lookup tableand for use in identifying and managing individual network flows 510. Inaddition to identifying a connection in this manner, the firewall 504may communicate the corresponding firewall action (e.g., selectingsecurity processing for a network flow, redirecting a network flow, ordropping a network flow) to the offload module 512 for storage in thelookup table and for use by the offload module 512 in managing networkflows 510.

FIG. 6 shows a method for operating a firewall system. The method 600may generally include providing a first path for network traffic througha firewall on a host device as shown in step 602 and providing a secondpath for network traffic through an offload module as shown in step 608,where the second path is separate from the first path. It will beunderstood that although these paths are illustrated as being providedsequentially, in general, both of these paths remain continuouslyavailable for network traffic, and which of the paths is assigned to aparticular network flow may depend on the current status of the networkflow, among other factors.

Thus, in one aspect, the method 600 may include directing a network flowincluding one or more packets along the first path to the firewall andapplying one or more firewall rules as shown in step 604. In response todetermining with the firewall that the network flow is permitted by theone or more firewall rules, the method may include communicating aninstruction from the firewall to the offload module for the offloadmodule to handle packets for the network flow, thereby directing thenetwork flow from the firewall to the offload module.

As shown in step 610, while the network flow is directed through theoffload module (or more generally, the network processing unit or othervirtual path outside the firewall), the method may include determiningwith the offload module that the network flow handled by the offloadmodule is not valid, invalidating a state stored by the offload moduleas corresponding to the network flow, and returning the network flow tothe firewall. In one aspect, the offload module may include a kernelspace process executing on the host device for the firewall. Thus, thehost may provide a virtual path for network connections that offloadsfirewall processing as described herein without requiring separatenetwork processing hardware. In another aspect, the offload moduleincludes a process executing on a network processing unit for thenetwork traffic. Invalidating the state may include invalidating oneconnection handled by the offload module, a group of level threeconnections handled by the offload module, or all connections handled bythe offload module.

Although not illustrated in FIG. 6, it will be understood that anintrusion prevention system may analyze a network flow independentlyfrom the firewall and the offload module. In response to determiningwith an intrusion prevention system, such as one executing in a userspace on the host device, that the network flow handled by the offloadmodule presents a security risk, the method 600 may include remediatingthe network flow. For example, remediating the network flow includesreturning the network flow to the firewall. Remediating the network flowmay also or instead include disconnecting the network flow. Remediatingthe network flow may also or instead include remediating a source or adestination of the network flow.

In another aspect, the method may include providing a first path fornetwork traffic through a firewall on a host device, providing a secondpath for network traffic through an offload module, the second pathseparate from the first path, and switching a network flow between thefirst path and the second path based on one or more of a group offirewall rules, a group of intrusion prevention rules, and a group ofpacket validity rules. The network flow may include one or more packets,the group of firewall rules may cause a transition of the network flowfrom the first path to the second path. The group of intrusionprevention rules may cause a transition of the network flow from thesecond path to the first path. The group of packet validity rules maycause a transition of the network flow from the second path to the firstpath.

Additional details of firewall offloading methods and systems are nowprovided. In one aspect, the system may generally support firewalloffload from a host Linux system to a Network Processor Unit (NPU). Inthis context, the system may cache and offload network security andforwarding operations from a Linux networking stack to an offload modulethat can subsequently perform them while making end-to-end behaviortransparent to the networking stack. In one aspect, the offload modulemay be an optimized or computationally efficient software implementationexecuting on the host device. In another aspect, the offload module mayinclude additional hardware such as a special-purpose network processorcapable of applying the operations at high traffic rates and lowlatency. The system may also support high-speed bypass of the networkingstack through user-space security functions such as IPS, Web filtering,SSL inspection or AV scanning.

The platform may include, for example, a commercial off-the-shelfnetworking stack (i.e., Linux) capable of applying forwarding andsecurity decisions statefully at a connection-oriented level, a set ofabstract cached mappings of traffic to operations (“flow cache”), amirror copy of networking stack state which the offload modulesynchronizes with the network stack, mechanisms in the networking stackto select traffic that can reliably be mapped to operations, aninterface (API) for programming and querying flow cache and the mirrorednetworking state in the offload module and to invalidate and/or updatethat cached state with that of the networking stack, an interface (API)for the offload module to deliver traffic that misses the flow cache tothe networking stack, a high-speed RPC mechanism to implement APIoperations to hardware offload module implementations without undulydelaying packet processing (see RPC patent proposal), a high throughputtransport for packets to and from user-space security applications, anda per-connection state table (“Unified Session Table”) that augments thenetwork connection state with security state to allow securityapplications access to security-relevant information at the latency ofmemory access.

As a significant advantage, this architecture can augment commercialoff-the-shelf networking stacks to support hardware and/or softwareacceleration of the firewall as well as network stack bypass to inlinesecurity functions. The acceleration may generally lower cost andimprove performance, while the network stack bypass advantageouslyfacilitates scalable addition of inline security functions withoutbottlenecking performance in the networking stack.

The remainder of this disclosure refers to the host Linux networkingstack as Path 1 (the SP) and the offload module as Path 2 (the FP). Ifthe FP is a kernel module in the host system itself, it is referred toas the Virtual FP (VFP) while the offload module that resides on aseparate Network Processing Unit (NPU) is referred to as XFP.

Path 1 (SP) may process the initial packets of a new flow, populatingthe Linux conntrack table as these initial packets pass through all thenetfilter/iptables hooks registered in the host system. Once the SPmakes a firewall decision for a particular flow, it can then offloadthat flow to Path 2 (FP) based on the firewall state for that flow inthe host Linux kernel by programming the FP using a well-defined Path 2Programming API. The firewall state cached on the FP is essentially abidirectional flow cache based on the digested version of the firewalland forwarding state that is available on the SP.

The FP Programming API may define a common interface to the SP to updateand query the forwarding state in the FP module as well as an interfacefor the FP to update SP firewall state. The FP API may provide twogroups of API functions. The first group, Path 1 to Path 2 (SP2FP) APIallows the SP to update and query the firewall state on the FP. Thesecond group, Path 2 to Path 1 (FP2SP) API, on the other hand allows theFP to update relevant state information on the Linux kernel, namely perport and connection traffic statistics.

The SP may use the SP2FP API from three different contexts: (1) Duringsystem initialization to program the initial FP state including a FPrepresentation of all the network ports in the system as well asrelevant state for direct packet delivery mechanism to the host systemuserspace (whether it is via DPDK in case of XFP or netmap in case ofVFP); (2) Through various points in the packet-processing context in theLinux network stack when the SP decides to offload a flow to the FP ordiscovers new L2 neighbors or L3 routes; and (3) As a response tochanges in the overall system such as configuration changes that canimpact firewall state (firewall rules added, modified, deleted,interface configuration changes etc.) or changes in the dynamic firewallstate (updates to the routing table, neighbor table, users logging inand out, etc.).

The FP offload module may use the FP2SP API functions to: (1) Update perport and per connection traffic statistics in the host system Linuxkernel; and (2) Hand terminating connections in bulk to the host systemLinux kernel.

The architecture may generally keep the interaction between SP and FPagnostic to the type of the FP that is running with a well-defined FPAPI as described earlier. The SP may effectively call same API in thesame manner whether the FP is VFP or XFP, with the API hiding thedifferent mechanisms that are in place to access or update the FP statefrom the SP. In case of VFP, the API may access the FP state through adirect memory read/write while XFP utilizes a high-speed, PCIe basedRemote Procedure Call (RPC) mechanism to access memory on the NPU. Incase of XFP, the requests issued by the FP offload module may issuerequests to invoke FP2SP API functions on the host system through theuse of well-defined control messages sent from the NPU to the hostsystem via a PCIe based control message channel that is establishedbetween the FP and SP. A Control Message Handler (CMH) on the host maylisten to these control messages from the FP and multiplex the FPrequests to the SP by calling the appropriate FP2SP API function.

Architecturally, a host device as described herein may include: (1) Path1 (SP) using a Linux kernel networking stack, conntrack, iptables,netfilter, etc.; (2) an FP API module using a Linux kernel module thatimplements the FP API (SP2FP and FP2SP); (3) one or more instances ofIntrusion Prevention System (IPS) or other software applications runningin userspace; and (4) a Unified Session Table—A per-connection statetable that augments the network connection state in the host Linuxkernel with security state to allow security applications running inuserspace to access security-relevant information at the latency ofmapped memory access. In case of XFP, the host device may furtherinclude: (1) a Remote Procedure Call (RPC) module that allows for a highperformance and scalable access to the NPU memory for use by the FP APIimplementation; and (2) a Control Message Handler (CMH) that processescontrol messages from XFP for bulk connection traffic statistics andconnection termination indications that will invoke relevant FP2SP APIfunctions to update conntrack table state in the host Linux kernel or tohand connections over. In the case of VFP, a Linux kernel FP module thatoffloads firewall, routing and forwarding logic from the Linuxnetworking stack

In case of XFP, the following system components are present on a NetworkProcessing Unit (NPU): (1) an RPC request handler running in the NPUkernel; (2) an FP API request handler running in the NPU kernel; (3) aPath 2 module including a packet processing module that runs on the NPUuserspace that offloads packet processing on the host kernel to free upprocessing cycles for other code running on host processor

The system may maintain a number of tables to facilitate management oftraffic offloading. There are a number of different types of tables thatmay be present in the FP for offloading traffic that the SP programsusing the SPP2FP API and the FP offload module consults as it processespackets.

Logical Interface Table

A logical interface (LIF) table may determine the incoming interface ofa given packet. It maps a physical input port along with possibly a VLANtag (if present in the packet) to a logical interface. Each entry in theLIF table contains information including forwarding mode of the LIFwhich determines whether the FP should treat packets received over thisLIF in bridge or routing mode, L2 address (MAC) of the correspondingport and the MTU.

Microflow Table

A Microflow table may provide a flow cache that allows the FP offloadmodule to associate a received packet to a flow. In addition to thetraditional 5-tuple connection key, i.e., layer 3 (IPv4 or IPv6) sourceand destination address, layer 4 protocol identifier, L4 source anddestination port, the flow key also contains ingress LIF identifier,Ethernet type, source and destination MAC address in the packetreceived. A microflow entry essentially reflects the cached version ofthe firewall decision and actions that are associated with a flow asdetermined by the initial processing of the first few packets by the SP.Once Path 1 decides to offload a flow, it programs a connection entryand microflow entries (one per direction) with information that allowsthe FP to disposition the packet including: (1) Action for packetdisposition, i.e., whether to send it out to a port or to a securityapplication such as IPS, Web filtering, SSL inspection or AV scanningthat resides on the host userspace, and (2) Indices to access thecorresponding FP connection and NHOP entry for connection tracking andforwarding information. If there is no usable microflow entry for apacket received, FP adds a new microflow entry in the table, marks it as“incomplete” and sends the packet to host Linux kernel SP forprocessing. When the SP decides to offload the corresponding flow atsome point (in the context of processing a packet), it uses the SP2FPAPI to complete the programming of this microflow entry and programs acorresponding Path 2 connection entry (as described in the next table).

Connection Tracking Table

A connection tracking table may store state information associated witheach bidirectional, 5-tuple based connection in the system. The mostcommon use for this information is to store TCP sequence and stateinformation in order to perform proper TCP connection tracking as wellas source and destination NAT. This is required for stateful firewallsemantics.

While the XFP maintains a separate table for connection trackingpurposes, the VFP on the other hand may dispense with this table becausethe VFP resides in the host Linux kernel and can access the Linuxconntrack table for all connection tracking purposes. The SP programs anentry in this table at the time it decides to offload a flow.

Data stored per connection entry in the FP includes the following (Notein case VFP, there is no separate connection table and the followingdata is part of the SP Linux kernel conntrack entry):

-   -   Connection state    -   In case of a TCP connection:        -   TCP sequence info per direction        -   Packet retransmit counter (for maximum packet retransmit            tracking)        -   FIN tracking information in case connection termination on            the FP is enabled for bulk connection handover optimization    -   Source and Destination NAT configuration    -   IPS verdict (cut-thru or drop)    -   PCIe VF identifier that connects to the IPS instance    -   Connection traffic stats

The SP may manage the connection timeouts for all connections includingoffloaded ones. For the existing timeout mechanism in the SP to workproperly for connections offloaded to the FP, there is a need to keepthe timestamp and the traffic stats in the host Linux kernel conntracktrack up to date. For the VFP case, the FP updates the host Linux kernelconntrack directly using a function call to the appropriate FP2SP APIfunction. For the XFP case, the FP offload module sends periodic batchupdates in a control message containing traffic statistics for theoffloaded connections. There is a control message handler in the hostsystem that listens to such control messages to update the Linux kernelconntrack entry and set the entry timestamp to the current time usingthe appropriate FP2SP API function. A benefit of keeping the connectiontraffic statistics on the host Linux kernel conntrack is that thisallows all the Linux based conntrack tools to be available to the userfor debugging.

NHOP Table

An NHOP table is programmed with each L3 router or L2 neighbor on agiven port. A NHOP entry essentially is a digested version of theforwarding information in the SP. L3 NHOP entries are for routed trafficwhile L2 entries are for bridged traffic. In case of L3, in addition tothe outgoing port information, the NHOP entry also stores the source anddestination MAC address that the offload module will overwrite beforesending the packet out. The FP offload module consults this table as thelast step using the NHOP index stored in the microflow entry. The SPprograms or updates entries in the NHOP table from appropriate points inthe Linux networking stack when it discovers new L2 or L3 neighbors.

At a high level, packets that match an offloaded flow with fullyprogrammed firewall state on the FP may completely bypass the SP on thehost and get directed either out to a network port or delivered directlyto one of possibly multiple instances of Intrusion Prevention System(IPS) software application running in the host userspace. The FP maydeploy an appropriate direct delivery mechanism to deliver packets tothe host userspace efficiently, whether it is netmap in the VFP or DPDKin the XFP case. When packets do not match an offloaded flow or a flowwith incomplete firewall state, FP may send them to the SP using awell-defined packet API. This API allows the FP offload module to insertrelevant meta-data to the start of the packet for use by the driver onthe host to associate the packet with the Linux netdev that correspondsto the port on which NPU received the packet.

When the system receives a packet from a network port on the NPU, the FPoffload module processes it first. The FP offload module may performvarious validity checks on the L2, L3 and L4 headers in the packet. Ifany of these checks fail or if the packet corresponds to a type oftraffic that does not qualify for offload such as broadcast or multicasttraffic, the FP offload module sends the packet to the host system forprocessing by the firewall.

If the traffic qualifies for offload, using the ingress port ID andoptionally the VLAN ID (if present in the packet), the FP offload modulelooks up the LIF table. If the lookup does not return a valid LIF entry,FP offload module drops the packet. If there is a valid LIF entry butoffload is disabled (on that particular LIF or for all LIFs), the FPoffload module sends the packet to the host system, inserting ameta-data before the packet that will allow the network driver on thehost system to associate the packet with the Linux netdev thatcorresponds to the ingress port.

If there is a valid LIF and offload is on, the FP offload module looksup the Microflow table next. The first time FP offload module sees apacket for a new flow; it inserts a “pending” microflow entry into thetable and sends the packet to the host system. This time, in addition tothe metadata that identifies the received port, the offload moduleinserts another metadata, a microflow ID to indicate to the SP that theflow that corresponds to this packet is a candidate for offload. The SPprograms microflow entries using a kernel hook inserted towards the endof the Linux network stack packet processing. The SP will offload a flowwhen it meets the following criteria:

-   -   The flow is an offload candidate—FP offload module indicates        this to the SP on the host system by passing a microflow ID as        metadata along with the packet. The offload module will only        indicate candidate offload flows if the traffic type is        supported for offload.    -   All the firewall/forwarding decisions for the packet have been        made by the Linux network stack (IPS, L2/L3 forwarding, NAT        etc.)    -   The conntrack entry related to the flow is in established state    -   The flow is between NPU ports    -   The flow's NHOP is resolved if this is an L3 flow    -   The flow is not destined to a local address, in other words        connection is not terminated locally    -   The connection table on the FP is not full

When the conditions above are met, the SP will program the correspondingconnection and microflow entry in the FP, using the appropriate state inthe Linux kernel, including state in the iptables, netfilter, conntrack,and bridge and routing tables to offload the flow to the FP. Onceoffloaded, if for any reason SP receives a packet that matches thatflow, the SP has to consult the FP for validity of the connection stateas well as TCP sequence state for TCP flows using the SP2FP API. The SPalso programs the UST entry corresponding to the connection to indicatethe flow's offloaded status. This enables any host processing modulessuch as IPS to know that they can inject or transmit traffic for thesame traffic directly to Path 2.

The FP offload module will continue sending packets to the host systemas long as it matches a ‘pending’ microflow entry. Once the microflow isprogrammed, the FP offload module lookup will yield an ‘active’microflow entry which contains information that the FP offload moduleuses to determine how to disposition the packet as well as an index tothe corresponding connection entry.

Using this index, the offload module loads the connection entry. If thepacket is a TCP packet, it performs connection tracking (sequencechecking and maximum retransmit tracking) using the state in theconnection entry. If TCP sequence checking fails, the offload modulesends the packet to the host system for the Linux network stack toprocess it. If the connection entry indicates an IPS verdict to drop (asa result of earlier packet inspection that was performed by the IPSmodule), the offload module drops the packet without any furtherprocessing.

-   -   Based on the action indicated in the microflow entry, the        offload module can direct the packet to:    -   Another network port (FWD Action)    -   To an IPS instance running on the host system userspace before        forwarding (IPS Action)    -   Drop the packet (DROP Action)

If the connection entry indicates that NAT should be performed for apacket, the offload module will modify the contents of the packet topossibly overwrite IP source/destination address, L4 source/destinationport depending on whether DNAT and/or SNAT is to be performed. In casethe microflow action is to send the packet to IPS, offload applies DNATto the packet, recomputing L3 and L4 checksums in the headers before itsends the packet to IPS. SNAT on the other hand is applied to the packetbefore it is sent out on the network (and after it is received from IPSif it had been sent to IPS earlier). If the microflow action is “IPS”,the offload module retrieves the PCIe VF information to reach theappropriate IPS in the host system from the connection entry. Beforesending the packet to the IPS, the FP offload module caches the ID ofthe microflow in a meta-data at the start of the packet. After thepacket inspection and if IPS decides to let the packet through, IPSretains the metadata cached when it sends the packet back to FP. Thisway, the offload module can efficiently use the previous lookup resultsfor continuing the packet journey. At some point during the inspectionprocess performed by the IPS module, it can reach a verdict that may beto drop or bypass inspection (i.e., cut-through verdict) for allsubsequent packets corresponding to the flow. When that happens, the IPSmodule updates the universal session table entry correspondingconnection entry in the host system Linux conntrack. This then resultsin an SP2FP API call to the FP that will in turn update the IPS verdictstored in the FP connection entry.

Whether the offload module sent the packet to IPS earlier or not, if thepacket needs to go out to the network, the offload module may consultthe NHOP table. Using the NHOP identifier cached in the microflow entry,the FP offload module may access the NHOP table entry to retrieve theoutgoing port information. Additionally, in case the NHOP is an L3 NHOP,the offload module may overwrite the source and destination MACaddresses in the packet with those cached in the NHOP entry, decrementthe TTL value in the IP header and re-compute IP header checksum. Thisis also the stage when the offload module would apply SNAT to the packetif the connection entry dictates so, potentially updating the L4 headerwith a different source port value and requiring a layer checksumre-computation. Once the offload module makes all the necessarymodifications to the packet, it can send the packet out on the portindicated in the NHOP entry.

The FP module may also support packet injection by the IPS module. Forpackets IPS module injects to the network, the module may insertmeta-data with corresponding identifiers to allow the FP module tolookup information for processing and sending the injected packet outthough an NPU interface.

When the FP offload module detects a TCP connection termination, itmarks the corresponding connection entry as ‘reclaim pending’ and sendsany further packet received for that flow to the host system for the SPto process the connection termination. As part of the connectiontermination, the SP may call an SP2FP API that returns all the cachedconnection tracking state including TCP sequence state used for thehandover of the connection from FP to SP. In order to avoid an API callfor each terminating connection, the FP offload module may alsoimplement a ‘bulk’ connection reclaim method. In this case, the offloadmodule on the FP processes the connection terminations (FIN tracking)and periodically sends a batch of connection termination requests to theCMH on the host system.

FIG. 7 shows an architecture for a firewall. In general, the firewall702 may be any of the firewalls described herein, and may have anarchitecture for offloading certain traffic management functions and/ornetwork flows to a network processing unit or the like associated withthe firewall 702. The firewall 702 may be hosted on a first processor704 such as an x86 architecture processer, which may include a kernelspace 706 hosting a client 708 that uses a first memory 710 to exchangemessages with another processor. The firewall 702 may be coupled througha local bus 720 such as a PCIe bus or any other suitable local hardwarebus to a second processor 734 such as a network processing unit with asecond kernel space 736 hosting a server 738 that uses a second memory740 to exchange data with the first memory 710 of the first processor704 so that the second processor 734 can host procedures that supportoperation of the first processor 704. It will be understood that, whilethe following describes the use of such interprocessor procedure callsto support operation of a firewall 702 that offloads certain traffichandling operations to a network processing unit, such as the firewallsdescribed herein, these techniques may more generally be used in anycontext where distributed processing is performed, e.g., where networkprocesses or other procedures or the like are offloaded from oneprocessor to another processor coupled through a local hardware bus.

As generally illustrated in FIG. 7, arguments for an interprocessorprocedure call may be provided by the client 708 to the first memory 710of the first processor 704, which may, for example, include directmemory access (DMA) memory or any other memory configuration suitablefor permitting remote access to memory of the first processor 704. Thisdata may be exchanged with an external resource such as the networkprocessing unit 734 via a direct memory access read and write to thesecond memory 740, or any other technique suitable for sharing databetween the two memories 710, 740. In general, after a DMA read orsimilar operation to transfer procedure arguments to the second memory740, the client 708 may send an interrupt to the server 738 that hoststhe remote procedure, and the server 738 may process the procedure callbased on data in the second memory 740. Results of the procedure maythen be returned to the first memory 710 via a DMA write or any othersuitable operation, and the client 708 may periodically poll acorresponding local memory location, e.g., in the first memory 710, forresults of the procedure executed by the server 738 on the secondprocessor 734.

This distributed approach to processing advantageously permits scalableexecution and parallelization of data transport and execution fordifferent network flows. While existing techniques for remote procedurecalls typically operate on distributed systems accessible via layer 4communication protocols, the techniques described herein mitigate thelatency involved with such communications that might otherwise diminishthe performance of or outright prevent inline network offloadingapplications. More specifically, the advantages to the techniquesdescribed herein include the creation of a low latency interface betweenan x86 processor and a locally coupled network processing unit, theability to execute procedure calls between processors inline within thepacket processing context in the x86 kernel, and a scalable executionmodel the supports parallel execution of separate network flows.Additionally, the techniques described herein provide an asynchronousinterface for procedures that do not require any response. That is,callers of the remote procedure may advantageously elect to executeeither a synchronous blocking or asynchronous non-blocking procedurecall without any reconfiguration of the transport mechanism.

In general, the protocol between the client 708 and the server 738 isimplemented as a multi-producer, single-consumer ring. The callingmechanism may make use of a shared memory location, located on the NPUlocal memory and accessible via the PCIe Base Address Register (BAR)from the x86. This shared memory location may contain the generalprocedure call state, along with the individual ring state, e.g., aconfiguration state, producer/consumer indices, and message descriptors.Whenever the client 708 adds a new message to the ring, the client 708may populate a descriptor for the entry and then the hardware for theserver 738 can be signaled that a new message is available forprocessing via an interrupt. For hardware without interruptcapabilities, the server may execute a server thread that can poll thering state and/or status instead. This calling mechanism can be scaledfor performance by adding more server threads. There may be separaterings provided for each independent handler thread and another separatering provided for low priority procedure calls. Each server 738 orserver thread may operate independently.

FIG. 8 illustrates interprocessor procedure calls stored in a sharedmemory. Initially, a client 708 initializes the client threads byconfiguring the procedure call state and incrementing the configurationversion field. This data may be stored in the shared memory locationusing a structure illustrated in the figure. Each message passed by theclient 708 will contain some metadata contained in a descriptorstructure (cmd_bar_desc). This metadata will contain the pointer to thex86 memory location or other direct memory access location or the likecontaining the procedure arguments and the length of the arguments. Theserver 738 may initiate a direct memory access operation or the like tocopy the procedure arguments in the descriptor to local memory for theserver 738. As part of this message payload, the client 708 may prependanother descriptor (cmd_buf_desc) that stores an identifier for theprocedure to be executed, a maximum size of the arguments that can bereturned, and any other information necessary or helpful for the server738 to handle the interprocessor procedure call.

The server 738 may execute the requested procedure locally using anyarguments passed by the client 708 as describe above. If there are anyreturn arguments, the server 738 may initiate another direct memoryoperation (e.g., a direct memory write) or the like to copy the databack to client 708 before signaling the completion of the procedurealong with the response descriptor (resp_buf_desc) prepended to thereturned data. The server 738 may also usefully append a magic value tothe end of the buffer, e.g., a unique value that the client 708 can useto detect when the transfer of return arguments is complete. Theresponse descriptor may also contain an instance of the magic value forevaluation by the client 708, along with an indication of the returnpayload length.

FIG. 9 illustrates descriptors for an interprocessor procedure call.More specifically, FIG. 9 illustrates a descriptor 902 from a client,such as the client 708 described above (also referred to as a provider)and a response descriptor 904 from a server, such as the server 738described above (also referred to as a handler) exchanged through theshared memory, such as the shared memory 740 described above. Theseparation of descriptors (e.g., descriptor 902) into three differentportions as illustrated advantageously permits optimization of thetransfer from a network processing unit for the server 738 to the client708 in a manner that avoids PCIe reads, which may otherwise addsignificant latency to the exchange of the messages. In one aspect, eachprocedure call may include an explicit indicator of priority or animplicit indicator of priority (e.g., based on the type of procedure).The server 738 may execute low priority calls in an interruptiblecontext while executing high priority calls as soon as possible inhigher priority processing context. In another aspect, each server 738thread may be separated out to a different processing core to improvelatency.

FIG. 10 is a flow chart of a method for an interprocessor procedurecall. This may generally include executing an inteprocessor procedurecall between two processors coupled by a local hardware bus as generallydescribed herein.

As shown in step 1002, the method 1000 may include storing one or morearguments for the interprocessor procedure call in a first memory of afirst kernel of a first processor. The first processor may for exampleinclude a general processing unit such as an x86 architecture processoror the like executing on a firewall that manages network traffic for anenterprise network. More generally, the first processor may be anyprocessor that might advantageously offload processing to one or moreother processing units locally coupled to the first processor via ahardware bus or the like capable of sharing memory in a configurationthat can reduce interprocessor messaging as described herein. In oneaspect, the hardware bus may include a PCIe bus conforming to thePeripheral Component Interconnect Express standard.

As shown in step 1004, the method 1000 may include storing a descriptorfor the interprocessor procedure call in the first memory. Thedescriptor may be any of the descriptors described herein, and mayinclude data to facilitate processing of the procedure such as anidentifier for a procedure requested in the interprocessor procedurecall and a location for the one or more arguments in the first memory.The descriptor may also or instead usefully specify a length of the oneor more arguments, and/or a maximum length of return arguments permittedfor the interprocessor procedure call. In general, the procedurespecified in the descriptor may be any procedure that might usefully beperformed by a network processing unit that supports operation of afirewall, e.g., by managing network flows for the firewall, or any otherprocedure that might usefully be offloaded to a second processorconnected via a local hardware bus to a first processor and sharingmemory with the first processor.

Storing the descriptor may include storing the descriptor in a ringbuffer or any other suitable memory structure or the like. Where a ringbuffer is used, the method 1000 may include incrementing an index of thering buffer for each interprocessor procedure call requested by thefirst processor. While a ring buffer provides a useful data structurefor storing and exchanging sequences of procedure calls, any othersuitable memory structure or technique may be employed, preferably thatmaintains low latency be reducing or eliminating explicit interprocessormessaging traffic.

As shown in step 1006, the method 1000 may include reading thedescriptor from the first memory to a second memory of a second kernelof a second processor coupled to the first processor over the localhardware bus. The second processor may include a network processing unitexecuting on a network processor that shares management of networktraffic with the firewall, or otherwise supports operations of thefirewall or some other host device. Reading the descriptor may, forexample, include initiating a direct memory access read with the firstprocessor to transfer data from the first memory to the second memory.

As shown in step 1008, the method 1000 may include sending an interruptfrom the first processor to the second processor. The interrupt may be ahardware interrupt to the second processor indicating a request forexecution of the interprocessor procedure call. A hardware interruptprovides a useful, independent channel for communicating the presence ofa new procedure call from the client available for handling by a remoteserver, however any other suitable interprocessor messaging techniquemay be used, consistent with the desired performance of the distributedprocessing system. In one aspect, the interrupt may be a variableinterrupt used to signal, e.g., different priorities for differentprocedure calls, or to explicitly or implicitly communicate any otheruseful information associated with a call. For example, the interruptmay be selected by the first processor from among at least a lowpriority interrupt and a high priority interrupt, and the secondprocessor may preferentially process procedures associated with the highpriority interrupt over procedures associated with the low priorityinterrupt.

As shown in step 1010, the method 1000 may include processing thedescriptor with the second processor. This may occur in response toreceiving the interrupt at the second processor (e.g., at the server),and may include copying (e.g., with a direct memory access read orsimilar mechanism) the one or more arguments from the location in thedescriptor to the second memory of the second processor if this data hasnot already been transferred to the shared memory. In this context,copying the one or more arguments from the location may includeinitiating a direct memory access read from the location with the secondprocessor.

As shown in step 1012, the method 1000 may include processing the one ormore arguments in the second kernel of the second processor using theprocedure specified by the indicator. In general, the procedure may beany function, procedure, or the like executable by the second processoreither alone or in combination with other processing resources. In oneaspect, this may be a procedure supporting operation of the firewall orproviding management of network flows through a network processing unitaccording to firewall rules.

As shown in step 1014, the method 1000 may include writing a response tothe interprocessor procedure call from the second processor to alocation in the first memory of the first processor, such as thelocation in the first memory containing the one or more arguments forthe inteprocessor procedure call stored by first processor. The re-useof this location permits conservation of memory, and provides a specificlocation for the first processor to poll locally when waiting for aresponse to the procedure call. In one aspect, writing the response tothe interprocessor procedure call may include initiating a direct memoryaccess write from the second memory to the first memory.

The response may, for example, include a first instance of a uniquevalue at a first position in the response, where the first position isallocated to identifying a code for signaling a completion of theprocedure. The response may also include a second instance of the uniquevalue at a second position in the response, where the second position isallocated to signaling the completion of the procedure. In general, theunique value may be any binary sequence useful for signaling completionof the requested procedure. The unique value may be created by theserver after completion of the procedure so that the server can ensurethe unique value does not accidentally appear elsewhere in the responsedata, and then repeated so that the client can identify the unique value(e.g., based on the first instance) and verify completion of theprocedure (e.g., based on the second instance). While the unique valuemight naturally appear at a beginning and end of the response data, anyother predetermined location(s) within the response data may also orinstead be used. While a unique value provides a useful medium forsignaling completion of the procedure, other techniques may also orinstead be used for the response data to self-identify completion of therequested procedure, or to otherwise signal completion of the requestedprocedure in a manner consistent with the scalable, low-latencyrequirements of a particular implementation.

As shown in step 1016, the method 1000 may include polling the firstmemory of the client for a response to the interprocessor procedurecall. This may include polling the first memory for a change in datastored at the location specified for the response, or more specificallyfor the unique value used by the second processor to signal completionof the procedure to the first processor. Although illustrated asoccurring after the response is written from the shared memory, thepolling may begin any time after the descriptor is stored by the clientor transferred to the shared memory (or sooner, although no responsewould be expected). The polling may occur at any suitable frequency. Asa significant advantage, polling for a response on the client side,e.g., for the unique value indicating completion of the requestedprocedure, permits the client to receive a response to the requestwithout any need for explicit messaging (and the associated latency)between the client and server using the shared hardware bus.

According to the foregoing, there is further disclosed herein a systemfor interprocessor procedure calls. The system may generally include afirst processor, a second processor, and a hardware bus locally couplingthe first processor and the second processor in a communicatingrelationship. The first processor may include a first kernel with afirst memory accessible to external resources and the first processormay execute a client configured to request a procedure from a remoteprocessor with an interprocessor procedure call. The second processormay include a second kernel with a second memory, and may host a serverfor the interprocessor procedure call. The first processor may storearguments for the interprocessor procedure call in the first memory andsignal a request for the interprocessor procedure call with a hardwareinterrupt from the first processor to the second processor over thehardware bus. The second processor may be configured (e.g., by computerexecutable code stored in a memory) to read the arguments from the firstmemory of the second processor, to process the arguments according tothe procedure, and to write a response to the interprocessor procedurecall to the first memory of the first processor.

The first processor may, for example, be a general processing unit of ahost executing a firewall for an enterprise network. The secondprocessor may be a network processing unit processing network flows forthe firewall. The first memory may be a direct memory access memoryaccessible to external devices through the hardware bus withoutsupervision by the first processor.

FIG. 11 illustrates messaging for batched procedure calls. As a furtheroptimization of the techniques described above, multiple procedure callscan be transferred as a single direct memory access operation. In anembodiment, the server may copy calling arguments (or responses) formultiple calls within a larger contiguous data block using the datastructures illustrated in FIG. 11. In this manner, the server can accessto multiple payloads using any suitable limits on size or number such asup to the number of calls already queued, up to a boundary of a clientdata block, or up to a preset batch limit. To facilitate this protocol,the server may detect when there are additional calls available forprocessing by looking at the difference between the producer/head andconsumer/tail indices, as illustrated in FIG. 11.

This scheme allows batching of reads from and writes to the clientmemory, for example, so that a processor from a client may batch aplurality of interprocessor procedure calls and request processing ofthe plurality of interprocessor procedure calls with a single interrupt.Typically, a batch from the first processor to the second processor willbenefit from being as large as possible, whereas a batch from the secondmemory to the first memory may increase latency for a specific procedurecall while waiting for other called procedures to execute. The batchsize may be tunable, and may be different for each server platform. Inone aspect, a no-op descriptor may serve as a BAR descriptor with a flagset to indicate a NO-OPERATION to prevent the unused space in the DMAbuffer from being transferred. To use a single direct memory accessbuffer large enough for a single ring, the direct memory access buffermay be sized for all the descriptors, e.g., with a buffer size set tothe number of descriptors per ring times the maximum data size for acall. However, this may be too large as a practical matter, because manyprocedure calls have been observed to be significantly smaller than themaximum data size. Empirically, a factor of four appears to besufficient, suggesting a buffer size of about one fourth this maximumpotential size.

The above systems, devices, methods, processes, and the like may berealized in hardware, software, or any combination of these suitable fora particular application. The hardware may include a general-purposecomputer and/or dedicated computing device. This includes realization inone or more microprocessors, microcontrollers, embeddedmicrocontrollers, programmable digital signal processors or otherprogrammable devices or processing circuitry, along with internal and/orexternal memory. This may also, or instead, include one or moreapplication specific integrated circuits, programmable gate arrays,programmable array logic components, or any other device or devices thatmay be configured to process electronic signals. It will further beappreciated that a realization of the processes or devices describedabove may include computer-executable code created using a structuredprogramming language such as C, an object oriented programming languagesuch as C++, or any other high-level or low-level programming language(including assembly languages, hardware description languages, anddatabase programming languages and technologies) that may be stored,compiled or interpreted to run on one of the above devices, as well asheterogeneous combinations of processors, processor architectures, orcombinations of different hardware and software. In another aspect, themethods may be embodied in systems that perform the steps thereof, andmay be distributed across devices in a number of ways. At the same time,processing may be distributed across devices such as the various systemsdescribed above, or all of the functionality may be integrated into adedicated, standalone device or other hardware. In another aspect, meansfor performing the steps associated with the processes described abovemay include any of the hardware and/or software described above. Allsuch permutations and combinations are intended to fall within the scopeof the present disclosure.

Embodiments disclosed herein may include computer program productscomprising computer-executable code or computer-usable code that, whenexecuting on one or more computing devices, performs any and/or all ofthe steps thereof. The code may be stored in a non-transitory fashion ina computer memory, which may be a memory from which the program executes(such as random-access memory associated with a processor), or a storagedevice such as a disk drive, flash memory or any other optical,electromagnetic, magnetic, infrared or other device or combination ofdevices. In another aspect, any of the systems and methods describedabove may be embodied in any suitable transmission or propagation mediumcarrying computer-executable code and/or any inputs or outputs fromsame.

It will be appreciated that the devices, systems, and methods describedabove are set forth by way of example and not of limitation. Absent anexplicit indication to the contrary, the disclosed steps may bemodified, supplemented, omitted, and/or re-ordered without departingfrom the scope of this disclosure. Numerous variations, additions,omissions, and other modifications will be apparent to one of ordinaryskill in the art. In addition, the order or presentation of method stepsin the description and drawings above is not intended to require thisorder of performing the recited steps unless a particular order isexpressly required or otherwise clear from the context.

The method steps of the implementations described herein are intended toinclude any suitable method of causing such method steps to beperformed, consistent with the patentability of the following claims,unless a different meaning is expressly provided or otherwise clear fromthe context. So, for example, performing the step of X includes anysuitable method for causing another party such as a remote user, aremote processing resource (e.g., a server or cloud computer) or amachine to perform the step of X. Similarly, performing steps X, Y and Zmay include any method of directing or controlling any combination ofsuch other individuals or resources to perform steps X, Y and Z toobtain the benefit of such steps. Thus, method steps of theimplementations described herein are intended to include any suitablemethod of causing one or more other parties or entities to perform thesteps, consistent with the patentability of the following claims, unlessa different meaning is expressly provided or otherwise clear from thecontext. Such parties or entities need not be under the direction orcontrol of any other party or entity, and need not be located within aparticular jurisdiction.

It should further be appreciated that the methods above are provided byway of example. Absent an explicit indication to the contrary, thedisclosed steps may be modified, supplemented, omitted, and/orre-ordered without departing from the scope of this disclosure.

It will be appreciated that the methods and systems described above areset forth by way of example and not of limitation. Numerous variations,additions, omissions, and other modifications will be apparent to one ofordinary skill in the art. In addition, the order or presentation ofmethod steps in the description and drawings above is not intended torequire this order of performing the recited steps unless a particularorder is expressly required or otherwise clear from the context. Thus,while particular embodiments have been shown and described, it will beapparent to those skilled in the art that various changes andmodifications in form and details may be made therein without departingfrom the spirit and scope of this disclosure and are intended to form apart of the invention as defined by the following claims, which are tobe interpreted in the broadest sense allowable by law.

What is claimed is:
 1. A method comprising: providing a first path fornetwork traffic through a firewall on a host device; providing a secondpath for network traffic through an offload module, the second pathseparate from the first path; directing a network flow including one ormore packets along the first path to the firewall; applying one or morefirewall rules to the network flow; in response to determining with thefirewall that the network flow is permitted by the one or more firewallrules, communicating an instruction from the firewall to the offloadmodule for the offload module to handle packets for the network flow;and in response to determining with the offload module that the networkflow handled by the offload module is not valid, invalidating a statestored by the offload module as corresponding to the network flow andreturning the network flow to the firewall.
 2. The method of claim 1,wherein invalidating the state includes invalidating one connectionhandled by the offload module, a group of level three connectionshandled by the offload module, or all connections handled by the offloadmodule.
 3. The method of claim 1, further comprising, in response todetermining with an intrusion prevention system executing in a userspace on the host device that the network flow handled by the offloadmodule presents a security risk, remediating the network flow.
 4. Themethod of claim 3, wherein remediating the network flow includesreturning the network flow to the firewall.
 5. The method of claim 3,wherein remediating the network flow includes disconnecting the networkflow.
 6. The method of claim 3, wherein remediating the network flowincludes remediating a source or a destination of the network flow. 7.The method of claim 1, wherein the offload module includes a kernelspace process on the host device.
 8. The method of claim 1, wherein theoffload module includes a process executing on a network processing unitfor the network traffic.
 9. The method of claim 1, further comprisingmanaging the one or more firewall rules from a threat managementfacility for an enterprise network.
 10. The method of claim 1, whereinthe firewall is a kernel process executing on the host device.
 11. Amethod comprising: providing a first path for network traffic through afirewall on a host device; providing a second path for network trafficthrough an offload module, the second path separate from the first path;and switching a network flow between the first path and the second pathbased on one or more of a group of firewall rules, a group of intrusionprevention rules, and a group of packet validity rules.
 12. The methodof claim 11, wherein the network flow includes one or more packets. 13.The method of claim 11, wherein the group of firewall rules cause atransition of the network flow from the first path to the second path.14. The method of claim 11, wherein the group of intrusion preventionrules cause a transition of the network flow from the second path to thefirst path.
 15. The method of claim 11, wherein the group of packetvalidity rules causes a transition of the network flow from the secondpath to the first path.
 16. A system comprising: a firewall executing ina kernel space of a host device; an offload module executing on anetwork processing unit; a first programming interface for the firewallto redirect a network flow from the firewall to the offload module; asecond programming interface for the offload module to direct thenetwork flow from the offload module to the firewall; one or morefirewall rules stored on the host device and accessible by the firewallfor use in determining a firewall action for the network flow; and alookup table for the network processing unit, the lookup table storing alist of one or more connections directed through the offload module bythe firewall.
 17. The system of claim 16, further comprising anintrusion prevention system executing in user space of the host devicethe intrusion prevention system configured to detect potential threatsin the network flow.
 18. The system of claim 16, wherein the lookuptable identifies each of the one or more connections using at least anInternet Protocol source and destination address, a layer 4 source anddestination address, a Medium Access Controller source and destinationaddress, and a protocol identifier.
 19. The system of claim 16, whereinthe lookup table is used by the offload module to apply the firewallaction determined by the host device to the network flow.
 20. The systemof claim 19, wherein the lookup table includes a connection table. 21.The system of claim 19, wherein the firewall action includes at leastone of determining security processing for the network flow, redirectingthe network flow, or dropping the network flow.